International Journal of Instruction 
e-ISSN: 1308-1470 • www.e-iji.net 



July 2015 • Vol.8, No.2 
p-ISSN: 1694-609X 


An Evaluation of the Measurement of Perceived Classroom Assessment 
Environment 

Hussain Alkharusi 

Sultan Qaboos University, Oman, hussein5 @ squ.edu.om 


A classroom assessment environment is a classroom context experienced by 
students as the teacher determines assessment purposes, develops assessment tasks, 
defines assessment criteria and standards, provides feedback, and monitors 
outcomes (Brookhart, 1997). It is usually a group experience varying from class to 
class dependent upon the teacher’s assessment practices (Brookhart, 2004). As 
such, the measurement of class-level perception of the assessment environment 
should deserve recognition and investigation. This study aimed at evaluating the 
measurement of the perceived classroom assessment environment by comparing 
the psychometric properties of the scale at the student level and class level. Using a 
multi-stage random sampling process, data were collected from 4088 students 
nested within 236 classes of the second cycle of the basic education in the 
Sultanate of Oman. Students responded to the 18-items of Alkharusi's (2011) 
Perceived Classroom Assessment Environment Scale. Results of the principal axis 
factoring yielded two factors, learning-oriented and performance-oriented 
assessment environment, at both levels. However, the two factors explained about 
38% of the variance at the class level compared to about 20% of the variance at the 
student level. Reliability coefficients in terms of Cronbach alpha ranged between 
.79 and .83 at the class level compared to .65 and .67 at the student level. 

Key Words: validity, reliability, evaluation, measurement, classroom assessment, 
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INTRODUCTION 

Teachers’ assessment practices have been the focus of much attention for over the past 
years. However, significant interest in classroom assessment environment is a recent 
phenomenon. The classroom assessment environment represents an important 
component of the classroom climate (Brookhart & DeVoge, 1999). It involves exposing 
students to a variety of assessment practices created largely by the teacher (Brookhart, 
Walsh, & Zientarski, 2004). An issue that has received increased attention by 
educational assessment researchers is what do students think about the assessment 
practices that are used to evaluate their academic achievement. For example, Dorman 
and Knightley (2006) designed an inventory to measure students’ perceptions of the 
assessment tasks in terms of congruence with planned learning, authenticity, student 
consultation, transparency, and diversity. Several researchers have found that classes 
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using assessment tasks with high degrees of congruence with planned learning, 
authenticity, student consultation, transparency, and diversity tended to be associated 
with high levels of student self-efficacy, deep approaches to learning, and increased 
academic achievement (Alkharusi, 2013; Alkharusi, Aldhafri, Alnabhani, & Alkalbani, 
2013; 2014; Dhindsa, Omar, & Waldrip, 2007; Dorman, Fisher, & Waldrip, 2006). 
However, students’ perceptions of the nature of the assessment tasks used in the 
classroom represent part of the overall student assessment experience that leads up to the 
perceived classroom assessment environment. 

Perceived classroom assessment environment refers to the overall sense or meaning that 
students make out of the assessment practices by the teacher in the classroom (Brookhart 
& DeVoge, 1999). It is a classroom context experienced by students as the teacher 
determines assessment purposes, develops assessment tasks, defines assessment criteria 
and standards, provides feedback, and monitors outcomes (Brookhart, 1997). Based on 
achievement goal theory, Alkharusi (2011) developed a scale to measure students’ 
perceptions of the classroom assessment environment. The scale operationally defined 
the perceived classroom assessment environment as representing two dimensions: 
learning- and performance-oriented assessment environments. The learning-oriented 
assessment environment is characterized by classroom assessment practices that enhance 
student learning and mastery of subject materials such as asking students to do a variety 
of meaningful assessment tasks with moderate difficulty, providing them chances to 
improve their task performance, and giving them informative assessment feedback. The 
performance-oriented environment is featured by classroom assessment practices like 
asking students to do difficult and less meaningful assessment tasks with difficult 
assessment standards and criteria, emphasizing the importance of grades rather than 
learning, and comparing students’ performances to each other. Learning-oriented 
assessment environment was found to be positively associated with academic self- 
efficacy and academic achievement whereas performance-oriented assessment 
environment was found to be negatively associated with academic self-efficacy and 
academic achievement (Alkharusi, 2009; 2010; 2011). Also, female students tended to 
perceive their classroom assessment environment as being learning-oriented more than 
male students whereas male students tended to perceive their classroom assessment 
environment as being performance-oriented more than female students (Alkharusi, 2011; 
Meece, Herman, & McCombs, 2003). As such, measurement and evaluation of students’ 
perceptions of the classroom assessment environment deserve more recognition and 
investigation. 

When considering the items of Alkharusi’s (2011) scale, it appears that all of the items 
assessed the perception of the classroom assessment environment in reference to the 
“class experience” rather than to the “personal individual experience” (e.g., “In this 
class, students can find out their strengths”). Yet, Alkharusi (2011) measured perceived 
classroom assessment environment based on an individual student perception of the 
assessment environment rather than on a collective class perception of the assessment 
environment. Brookhart and her colleagues postulated that each classroom has its own 
“assessment ‘character’ or environment” perceived by the students and springs from the 
teacher’s assessment practices (Brookhart, 2004, p. 444; Brookhart & Bronowicz, 
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2003). Given that students within the same class are taught by the same teacher, it is 
critical to reexamine the measurement of the perceived classroom assessment 
environment at the class level to capture the shared common experience and thus 
perception of students within the same class about their classroom assessment 
environment. 

Purpose of the study and research questions 

Given the growing interest on students’ perceptions of the classroom assessment 
environment, research is needed to support the assertion that score interpretations from 
the available instruments such as Alkharusi's (2011) Perceived Classroom Assessment 
Environment Scale are conceptually accurate representations of the students’ 
perceptions of the classroom assessment environment. Therefore, the purpose of the 
current study was to evaluate the measurement of the perceived classroom assessment 
environment by comparing the psychometric properties in terms of construct validity and 
reliability of Alkharusi's (2011) Perceived Classroom Assessment Environment Scale at 
the student level and class level. The study was guided by the following research 
questions: 

1. What are the underlying dimensions of Alkharusi's (2011) Perceived Classroom 
Assessment Environment Scale at the class level versus at the student level? 

2. How does Alkharusi's (2011) Perceived Classroom Assessment Environment Scale 
perform psychometrically at the class level versus at the student level? 

METHODS 

Sample 

The sample for this study included 4088 students nested within 236 classes randomly 
selected using a multi-stage sampling process from the second cycle of the basic 
education grades from all governorates in the Sultanate of Oman. The number of 
participating students in each class ranged from 9 to 20 with an average of 17 students 
and a standard deviation of 3.66. Of all participating students, there were 2037 males 
and 2051 females. Of all participating classes, there were 116 male classes and 120 
female classes. 

Instrumentation 

The primary instrument in this study was the 18-items Perceived Classroom Assessment 
Environment Scale developed by Alkharusi (2011) to measure students’ perceptions of 
the classroom assessment environment. As reported in Alkharusi (2011), the items 
measured students’ perceptions of the classroom assessment environment on two 
dimensions. The first dimension was learning-oriented assessment environment (9 items; 
Cronbach’s a = .82; e.g., “In this class, students are given a chance to correct their 
mistakes”). The second dimension was performance-oriented assessment environment (9 
items; Cronbach’s a = .75; e.g., “In this class, the teacher compares students’ 
performances to each other”). Responses were obtained on a 5-point Likert scale 
ranging from 1 (strongly disagree ) to 5 (strongly agree). 
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Procedures 

A list of all students in the second cycle of the basic education grades from all 
educational governorates could not be obtained. As such, a multi-stage sampling was 
carried out to select the students for feasibility reasons. In the first stage, a simple 
random sample of 106 public schools (63 male schools and 43 female schools) was 
selected from a list of all schools (221 male schools and 147 female schools) maintained 
by the Ministry of Education. In the second stage, a random sample of two classes on 
average of the second cycle of the basic education grades was selected from each of the 
selected schools. This resulted in a sample of 4088 students situated in 236 classes. 

Then, permission was requested from the Ministry of Education and principals of the 
selected schools to collect data from the students of the selected classes during a regular 
class period one and a half month prior to the final school examinations. The students 
were informed that a study is being conducted to examine their perceptions of the 
classroom assessment environment. They were informed that they were not obligated to 
participate in the study, and if they wished to participate, their responses would remain 
confidential. They were also informed that their participation would not influence their 
grades or relations with the teacher of the subject being assessed. No student rejected the 
participation in the study. The students were asked to respond to the scale and to write 
their names to enable the author to match their responses with the grade received in the 
subject at the end of the semester. The final subject grades were obtained from the 
school administration. 

Statistical analyses 

In relation to the aforementioned purpose of the study and research questions, the 
following statistical procedures were employed: 

1. Student responses to each of the 18-items of the scale were aggregated to the class 
level. The result was a mean score for each class for each of the 18-items in the scale. 
Thus, two sets of data were developed based on the same scale items: student-level data 
and class-level data. 

2. The factor structure of the scale items at the student level and at the class level was 
examined by subjecting student data and class data to principal axis factoring analyses. 

3. The reliability was assessed by computing Cronbach’s alpha internal consistency 
reliability estimates for the student data and class data based on the resulting factor 
structure. 

4. Predictive validity was assessed by computing the Pearson product-moment 
correlation coefficients between the scales’ scores and the final grades received in the 
subject at the end of the semester at the student level and class level. 

5. Effect sizes for gender differences on the scales’ scores were computed at the student 
level and class level as evidence for construct validity. The effect sizes were computed 
based on multivariate analyses of variances. 


International Journal of Instruction, July 2015 • Vol.8, No.2 



Alkharusi 


49 


RESULTS 
Factor structure 

Two principal axis factoring analyses were separately performed on the student data and 
class data. At both levels, the initial unrotated solutions resulted in factor structures of 
two dimensions as suggested by the scree plot and eigenvalues exceeding unity. 
However, the unrotated factor structures were less meaningful and difficult to interpret 
due to multiple factor loadings. Therefore, the analyses proceeded to rotate the factor 
matrices orthogonally with varimax rotation. The analyses yielded two meaningful 
factors at both levels. The two factors were performance-oriented assessment 
environment and learning-oriented assessment environment as defined in the original 
scale developed by Alkharusi (2011). 

At the student level, the two factors accounted together 19.68% of the total variance. 
The first factor consisted of nine items of the perceived performance-oriented 
assessment environment accounting for 9.94% of the variance (eigenvalue = 1.79) with 
factor loadings > .30. The second factor consisted of nine items of the perceived 
learning-oriented assessment environment accounting for 9.74% of the variance 
(eigenvalue = 1.75) with factor loadings > .30. 

At the class level, the two factors accounted together 37.85% of the total variance. The 
first factor consisted of nine items of the perceived performance-oriented assessment 
environment accounting for 20.10% of the variance (eigenvalue = 3.62) with factor 
loadings > .33. The second factor consisted of nine items of the perceived learning- 
oriented assessment environment accounting for 17.75% of the variance (eigenvalue = 
3.20) with factor loadings > .40. 

Reliability 

At the student level, internal consistency coefficients for perceived performance- 
oriented and learning-oriented assessment environment were .67 and .65 as measured by 
Cronbach’s alpha, respectively. Perceived learning-oriented assessment environment 
correlated negatively with perceived performance-oriented assessment environment, 
r(4086) = -.28, p < .001. At the class level, the internal consistency coefficients for 
perceived performance-oriented and learning-oriented assessment environment were .83 
and .79 as measured by Cronbach’s alpha, respectively. Perceived learning-oriented 
assessment environment correlated negatively with perceived performance-oriented 
assessment environment, r(234) = -.40, p < .001. 

Predictive validity 

To evaluate predictive validity of the scale's scores, Pearson product-moment correlation 
coefficients relating final grades earned in the subject and the subscales’ scores were 
calculated at the student level and class level. At the student level, results indicated that 
the final grades earned in the subject correlated positively and significantly with the 
perceived learning-oriented assessment environment, r(4086) = .10, p < .001; and 
negatively and significantly with the perceived performance-oriented assessment 
environment, r(4086) = -.28, p < .001. At the class level, the results revealed that the 
class final grades earned in the subject correlated positively and significantly with the 
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perceived learning-oriented assessment environment, r(234) = .27, p < .001; and 
negatively and significantly with the perceived performance-oriented assessment 
environment, r(234) = -.44, p < .001. 

Construct validity 

Based on the classroom environment research showing that female students tend to hold 
more positive perceptions of their classroom environment compared to male students 
(e.g., Alkharusi, 2011; Meece et al., 2003), gender differences on the perceived 
classroom assessment environment scale scores were examined as evidence for construct 
validity. Multivariate analyses of variances were conducted at the student level and class 
level. Table 1 presents the means and standard deviations for males and females on the 
perceived classroom assessment environment subscales at both levels. 

At the student level, the results revealed statistically significant multivariate effects of 
student gender on the perceived classroom assessment environment, F( 2, 4085) = 
116.58, p < .001, Wilks’ Lambda = .05. Female students tended on average to perceive 
their classroom assessment environment as being learning oriented more than male 

students, F{ 1, 4086) = 71.79, p < .001, partial l]~ = .02. In contrast, male students 
tended on average to perceive their classroom assessment environment as being 
performance oriented more than female students, F(l, 4086) = 189.83, p < .001, partial 

if = .04. 

At the class level, the results revealed statistically significant multivariate effects of class 
gender on the perceived classroom assessment environment, F( 2, 233) = 22.47, p < 
.001, Wilks’ Lambda = .16. Female classes tended on average to perceive their 
classroom assessment environment as being learning oriented more than male classes, 

F( 1, 234) = 20.17, p < .001, partial l)~ = .08. In contrast, male classes tended on 

average to perceive their classroom assessment environment as being performance 

2 

oriented more than female classes, F(l, 234) = 38.48, p < .001, partial f) = .14. 


Table 1: Means and standard deviations for males and females on the perceived 
classroom assessment environment subscales at the student level and class level. 


Student-level 


Males (n 

= 2037) 

Females (n = 

2051) 


M 

SD 

M 

SD 

Learning-oriented environment 

3.59 

.61 

3.74 

.58 

Performance-oriented environment 

2.92 

.63 

2.64 

.67 

Class-level 


Males (n 

= 116) 

Females (n = 

120) 


M 

SD 

M 

SD 

Learning-oriented environment 

3.59 

.30 

3.76 

.27 

Performance-oriented environment 

2.90 

.32 

2.63 

.36 


DISCUSSION AND CONCLUSION 

The present study highlighted the importance and value of additional precision in the 
measurement of perceived classroom assessment environment. The study evaluated the 


International Journal of Instruction, July 2015 • Vol.8, No.2 



Alkharusi 


51 


psychometric properties of the measurement of the construct at the student level and 
class level. In summary, the results of the factor analyses yielded two factors, learning- 
oriented and performance-oriented assessment environment, at both levels. However, the 
two factors explained about 38% of the variance at the class level compared to about 
20% of the variance at the student level. Coefficients of reliability and validity at the 
class level were higher than at the student level. 

Overall, the findings pointed to a conclusion that perceptions of students about their 
classroom assessment environment are better tapped by a scale at the collective class 
level than at the individual student level. The perceived classroom assessment 
environment subscales at the class level demonstrate reasonable levels of construct 
validity, predictive validity, and reliability. Hence, these subscales may prove to be 
useful tools in helping teachers to identify classroom assessment practices targeted at 
enhancing student learning. 

These results supported the assertion that “classes have an assessment ‘character’ or 
environment” that originates from the teacher’s classroom assessment practices, and that 
“students construct their own meaning [of the classroom assessment environment] based 
in part on their group experiences” (Brookhart, 2004, pp. 444 - 445). In the current 
study, the classes were independent in the sense that each teacher taught only one class, 
and as such the results demonstrated that students’ perceptions of their classroom 
assessment environment did vary systematically across classrooms. These findings 
tended to confirm McMillan and Workman’s (1998, p. 29) conclusion that “Assessment 
and grading continue to be a private activity, with considerable variation among 
teachers.” In addition, the results highlighted the shared common experience and thus 
perception of students within the same class about their teacher’s classroom assessment 
practices. Therefore, the implication of the findings for future research is that 
researchers may need to consider not only the individual student perception of the 
classroom assessment environment which is referred to by Maehr and Midgley (1991, p. 
405) as the “psychological environment”, but also the aggregate perceptions of students 
in a class about their classroom assessment practices which is referred to as the 
“objective environment” (Church, Elliot, & Gable, 2001, p. 44). Specifically, when 
researchers are interested in the differential performances of classes as a result of teacher 
assessment practices, the class mean would effectively captures the variability among the 
classes. A class’s collective perception of the assessment environment would be a better 
predictor of the variation among classrooms in student-level achievement-related 
outcomes than an individual’s student perception. Future research might also need to 
consider how do the individual student perception about the classroom assessment 
environment interact with the collective class perception in influencing student 
achievement-related outcomes. Specifically, there might be a social persuasion effect in 
the classroom on an individual student perception about the teacher’s assessment 
practices which in turn might affect student achievement-related outcomes. From a 
socio-cognitive perspective (Coleman, 1990), when a student’s perception of the 
assessment environment are incongruent with the shared perceptions of the class, the 
student’s perceptions might be diminished. Future research in this area is needed to 
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further our understanding of both student and collective class perceptions of the 
classroom assessment environment. 
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Turkish Abstract 

Algilanan Simf-Ri Degerlendirme Ortamimn ORuniiinun Bir Degerlendirmesi 

Sinif-i 9 i degerlendirme ortami ogretmenler tarafmdan belirlenen degerlendirme amaci, gorevleri, 
kriterleri, standartlarmm, verilen doniitlerin ve izlenen 9 iktilarin barmdigi ogrencilerin 
deneyimledigi smif baglami olarak tammlanabilir (Brookhart, 1997). Bu genellikle simftan sinifa 
degi§en ve ogretmenlerin degerlendirme pratiklerine bagli bir grup deneyimidir (Brookhart, 
2004). Bu yuzden smif algi diizeyinin oRiimu ara§tirmaya deger bir konudur. Bu 9 ali!jma 
algilanan smif degerlendirme ortamimn oRumiinu ogrenci ve smif diizeyine gore kar§ila§tirarak 
incelemeyi ama 9 lamaktadir. Cok a§amah bir rassal omekleme sureci kullanarak, Umman’da temel 
egitimin ikinci basamaginda 236 sinifta ogrenim goren 4088 ogrenciden veriler toplanmiijtir. 
Ogrenciler 18 maddelik Alkharusi’nin (2011) Algilanan Smif Ri Degerlendirme Okegi’ne cevap 
vermi§lerdir. Faktor analizi sonu 9 larinda ogrenme-yonelimli ve perfonnans-yonelimli olarak her 
iki diizeyde de iki faktor ortaya 9 ikanni 5 tir. Fakat, faktorler smif duzeyindeki % 38’lik bir 
varyansi a 9 iklarken, ogrenci duzeyinde % 20’ lik bir varyansi a 9 iklami 5 tir. Smif duzeyinde 
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giivenirlik katsayisi .79 ve .83 arasindayken, ogrenci diizeyinde katsayi .65 ile .67 arasindadir. 

Anahtar Kelimeler: ge 9 erlik, giivenirlik, degerlendirme, 6 l 9 iim, sinif-i 9 i degerlendirme, ogrenci 
algisr 

French Abstract 

Une Evaluation de la Mesure d'Environnement devaluation de Salle de Classe Per9u 

Un environnement d'evaluation de salle de classe est un contexte de salle de classe experiment^ 
par des etudiants comme le professeur determine des buts d'evaluation, developpe des taches 
d'evaluation, definit des criteres d'evaluation et des nonnes, foumit le retour d'information et 
controle des resultats (Brookhart, 1997). C'est d'habitude une experience de groupe variant de la 
classe a la personne a charge de classe sur les pratiques d'evaluation du professeur (Brookhart, 
2004). En tant que tel, la mesure de perception de niveau de classe de l'environnement 
d'evaluation devrait meriter la reconnaissance et l'enquete. Cette etude a vise a evaluer la mesure 
de l'environnement d'evaluation de salle de classe per 9 u en comparant les proprietes 
psychometriques de l'echelle au niveau d'etudiant et le niveau de classe. En utilisant un processus 
d'echantillonnage aleatoire a plusieurs etages, les donnees ont ete rassemblees de 4088 etudiants a 
niche dans 236 classes du deuxieme cycle de la formation initiale dans le Sultanat de l'Oman. Les 
etudiants ont repondu l'a 18 articles d'Alkliarusi (2011) a l'Echelle d'Environnement devaluation 
de Salle de classe Per 9 ue. Les resultats de l'axe principal prenant en compte deux facteurs 
rapportes, environnement d'evaluation axe sur apprentissage et axe sur performance, aux deux 
niveaux. Cependant, les deux facteurs expliques environ 38 % du disaccord au niveau de classe 
compare a environ 20 % du disaccord au niveau d'etudiant. Les coefficients de fiabilite en termes 
d'alpha Cronbach se sont etendus entre .79 et .83 au niveau de classe compare a .65 et .67 au 
niveau d'etudiant. 

Mots Cles: validite, fiabilite, evaluation, mesure, evaluation de salle de classe, perceptions 
d'etudiant, environnement d'evaluation 

Arabic Abstract 

A a \ \ 

jlaj j t infill 0 ual^jc-l AjA^Li f\* oil ^ja Aailail AALuu ^All JjLuaII A_iL_^ail AIli 

(JiLaj saIxII ^ j .(Brookhart, 1997) ^c aja*j]I ^ja£s j jjjIxaj a 

jjli 111 j .(Brookhart, 2004) CjLuijLaa]! IaLuc .1 ^a (JjIjjj ajc. 1 _a^. s 

i^ja A^jAaSI AjL-all ^jjfill Ajjj (JjAjS AauIjaII dAA (■ Vlg'i ^)jA^. Aiu] (_gjluiA 

SAAxaa AjAxa!! ^Ia^JauIj (_g jlmAj t—llUall jIuia ^jaLiaaII Aj^)1a AjjtLo 

cjJ AilaLaiJ ^.uiLuiVI AiiiJl ^ja ^^auIjA (J*-a3 236 4-JUa 4088 O* 

(jc. AAulLul V 1 _jj\ a !1 .<£ jaJI \a^\ \ Lml Alkhamsi (2011) i> »jss 18 ^4° LJIJI 

4 > Q . ^ 1 1 AjJJj 4 *Q , 1 AjJJ ' ^ Q i I ^ a j i i ‘ j J ^ . 

% 20 v cJ*-aa]l (_gjlulA (Jjjt-jHil ^JA %38 ^>*-43 

jj ,9_i]Ua]l ^jIoia 0.67 j 0.65 s- 1 J** (JjLjia c _ 5 Ic. 0.83 j 0.79 ox ^ 

J 0.10 S- 1 AijlLo (_£ jIuia 0.44- J 0.27 (jjAj-ail £a JaUjjVI Cj^IaIxa 

k_l Aj^Isa jIulo 0.16 ( _3'A»-a] ^)Aoja£ ^jo'va\\ ^jIuia 0.28” 

i2*-aa]l jIuia AjL-all ‘^-44 AjUall d)l£ljAl TnoVl aAA .9_ilUall (_g jIuia ^ 5 ^ 0.05 

i^JljAlj 9_ilUa]l tilljAl jLjjC.VI A^,Vi AjaAI LJojI La£ > ^A_^all 9_i]Uall (_£ jIulo j^gic. 1 g >ulj3 ^ja J>>>^1 

.cr-> 0- -a^ail j 9_l]Uall AjxilA ^^3 JjA AjaI jA AjC- l*-« 

^ - -q *\ | Ajjj ^Ajlkll Alii jjj-aj f ^ Q * ^AAL-Lii 1 ^i ^Ax^all 'Aj-ujj^II 
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